Section: New Results

Markov Random Fields

Fusion of multitemporal and multiresolution remote sensing data and application to natural disasters

Participants : Ihsen Hedhli, Josiane Zerubia [contact] .

This work was carried out in collaboration with Prof. Gabriele Moser and Prof. Sebastiano Serpico from DITEN departement, University of Genoa, Italy.

Multitemporal data, Multiresolution data, Supervised classification, Hierarchical Markov random fields.

The capabilities to monitor the Earth surface, and especially urban and built-up areas, from environmental disasters such as floods or earthquakes, and to assess the ground impact and damage of such events play primary roles from multiple social, economic, and human viewpoints. In this framework, accurate and time-efficient classification methods are especially important tools to support rapid and reliable assessment of the ground changes and damages induced by a disaster, in particular when an extensive area has been affected. Given the huge amount and variety of data available currently from last-generation very-high resolution (VHR) satellite missions, (such as Pléiades, COSMO-SkyMed, or WorldView-2), the main difficulty is to develop a classifier that can take benefit of multiband, multiresolution, multidate, and possibly multisensor input imagery. In such a context, Markov random field (MRF) models are widely used to solve classification problems as they permit one to integrate contextual information into the classification scheme. Due to their non-causal nature, these models generally lead to iterative inference algorithms that are computationally demanding (e.g., optimization via simulated annealing), thereby justifying the choice of a hierarchical structure, with good methodological and application-oriented properties such as: (i) the causality in scale, under Markovian assumption, which allows the use of a non-iterative algorithm with acceptable computational time and (ii) the possibility to incorporate images acquired at multiple resolutions in the hierarchy for multiresolution and multisensor fusion purposes [10] . In the proposed method, multidate and multiresolution fusion is based on explicit statistical modeling through hierarchical Markov random field modeling. The model allows both input data collected at multiple resolutions and additional multiscale features derived through wavelets to be fused. The proposed approach consists of a supervised Bayesian classifier that combines: (i) a joint class-conditional statistical model for pixelwise information and (ii) a hierarchical MRF for spatio-temporal and multiresolution contextual information. Step (i) deals, first, with the modeling of the marginal statistics of the spectral channels acquired at each resolution and conditioned to each class. Step (ii) consists in the integration of this statistical modeling in a hierarchical Markov random field for each date. An especially novel element of the proposed approach is the use of multiple quad-trees in cascade (see Figure 1 ), each associated with each new available image at different dates, with the aim to characterize the temporal correlations associated with distinct images in the input time series and to support the joint analysis of multitemporal, multiresolution, and possibly multisensor imagery. The transition probabilities between scales and between different dates determine the hierarchical MRF since they formalize the causality of the statistical interactions involved [11] .

A multi-layer Markov model for change detection in temporally separated aerial image pairs

Participant : Josiane Zerubia [contact] .

This work was carried out in collaboration with Prof. Zoltan Kato from Institute of Informatics, University of Szeged, Hungary [http://www.inf.u-szeged.hu/~kato/ ], and Praveer Singh from Institut Mines-Telecom.

Multilayer Markov Random Fields (MRF), Histogram of Gradients (HOG), change detection, graph-cut optimization, aerial/satellite images.

In the proposed approach developed last year, we have tried to include both texture as well as pixel level information to build a three layer Markov model using the Histogram of Oriented Gradients (HOG) and the Gray Level Difference features on the topmost and bottommost layer respectively. Using a ground truth (GT) mask defined manually by an expert for each of the image pairs in the data set (obtained from the Hungarian Institute of Geodesy, Cartography and Remote Sensing), we employ a supervised technique to mark the initial set of pixels / sites as foreground or background. On the basis of the HOG difference and the Gray level difference feature vector corresponding to all the pixels in the image pair, a probability density function is fitted individually for the binary label set comprising of foreground and background labels using the GT. The probabilistic estimate is calculated using one training image pair for each data set. Using this probabilistic measure, a negative log likelihood is computed for each pixel (for both the features as well as the binary label set) which is then passed to the energy function of the proposed 3-layer MRF model. The final segmentation is obtained by minimizing the energy using a graph-cut algorithm, and subsequently a final foreground and background labelling is obtained over the combined layer. Figure 2 , shows aerial image pairs, one of them captured in 1984 by FOMI, Hungary (a) and the other one by GoogleEarth in 2007 (c). (b) is the ground truth and (d) is a combination of the hierarchical MRF based change detection (in red), ground truth (in green) and changes detected correctly (in yellow). This year, we have made a comparison of this method with two other multilayer MRFs for change detection developed at MTA-SZTAKI in Budapest, Hungary.

Graph-cut-based model for spectral-spatial classification of hyperspectral images

Participant : Yuliya Tarabalka [contact] .

This work has been done in collaboration with Aakanksha Rana (Institut Mines-Telecom/EURECOM).

Hyperspectral images, graph cut, multi-label alpha expansion, contextual information, energy minimization

The very high spatial and spectral resolution of the last generation of remote sensors provides rich information about every pixel in an image scene, hence opening new perspectives in classification, but also presenting the challenge of analysing high data volumes. While pixel-wise classification methods analyze each pixel independently, classification results can be significantly improved by including spatial information in a classifier.

In this work, we proposed a spectral-spatial method for hyperspectral image classification based on a graph cut [15] . The classification task is expressed as an energy minimization problem on the spatio-temporal graph of image pixels, and is solved by using the graph-cut $\alpha$-expansion approach. The energy to optimize is computed as a sum of data and interaction energy terms, respectively. The data energy term is computed using the outputs of the probabilistic support vector machines classification. The second energy term, which expresses the interaction between spatially adjacent pixels in the eight-neighborhood, is computed by using dissimilarity measures between spectral vectors, such as vector norms, spectral angle map, or spectral information divergence. The performance of the proposed method was validated on hyperspectral images captured by the ROSIS and the AVIRIS sensors. Figure 3 compares classification results obtained by applying support vector machines and the proposed approach for the ROSIS hyperspectral image acquired over the University of Pavia. The new method yields higher classification accuracies when compared to the recent state-of-the-art approaches.